The Universe of Academic Learning Outcomes: www.learning-outcomes.net

Learning outcomes of high quality are the basis of both, academic mobility and state-of-the-art curriculum design. Therefore a data base for learning outcomes on a European basis was implemented and is accessible since October 2010. The function of this web-based tool is to open the opportunity for European HE and CE institutions to publish intended learning outcomes (ILOs) of their study programs (on module level), to compare them to those of other institutions and thus to start a process of shared quality development of ILO descriptions and architectures. This paper presents the structure of the data base and discusses some of the problems not yet satisfyingly solved. Introduction to problems and proposed solutions To write transparent ILO descriptions in an internationally understandable way is associated with several difficulties. It begins with the absence of a common wording (even within one single language) and appropriate translations (into 23 official EU languages and those of future candidate countries like Turkey or Albania). It continues with the nonexistence of a European standard classification of education which could make it possible to find a single ILO in a pool of some ten thousands which exist in reality. And it ends with the painful high number of characteristics and metadata of intended learning outcomes which could be useful or even necessary for different purposes. As a means to meet some of the mentioned challenges the team of the VIRQUAL project has decided to establish a web-based platform, the ILO repository on www.learning-outcomes.net for the exchange of products, expertise and experiences in the field of writing learning outcomes, one of the core concerns the Bologna process and particularly of student mobility in both modes, physically and virtually. The expectation is that the communicative exchange itself will contribute to a more standardized wording. As language always and inevitably is developed by communication the users themselves (the scientific community) will generate a common language for writing and translating ILOs. The classification of ILOs is still a serious challenge composed of two aspects: on the one hand the particular features of learning outcomes, and on the other hand their allocation to particular study programs and modules. As solution for the first part of the problem we propose a numeric coding system for the well-defined description of essential ILO features. (A paper presenting a first draft was presented at the GMW conference 2010 in Zurich; Csanyi, 2010: Das ILO-Wiki.) Descriptors are cycles and abilities defined by the European Qualifications Framework as well as classes or domains (see Tippelt et al., 2003) and levels or stages of competences (see Dreyfus & Dreyfus, 1980). For the open question of classifying programs and modules a solution on a European level still has to be found. Different countries have different systems, if at all, which are not compatible with those of other countries. With the “Australian Standard Classification of Education (ASCED)” (Australian Bureau of Statistics, 2001) Australia, which is a much more homogenous continent in terms of history and politics, offers a practicable example. Canada contributes another elaborated classification (Statistics Canada, 2001). Although we cannot use them for European purposes in its existing form the users of learning-outcomes.net can utilize them to develop our own European folksonomy – which can in the best case become a forerunner of an official European document. This paper shows how the ILO repository (in the stage of version 1) is structured in response to the challenges discussed above, and how users (individual teachers as well as institutions) can contribute to and benefit from the common development of the (European) universe of academic learning outcomes – which could become the heart of the European Higher Education Area. The main objective of this paper is to invite all stakeholders to help with the final development of the drafted structure and the usability of the website. learning-outcomes.net – the global structure In the first attempt we planned a wiki (see Csanyi, 2010). But a wiki might be too little structured or even chaotic to provide the necessary retrieving functionality for high numbers of different objects with complex interdependences. Thus we decided to use a database instead and looked for a very simple one which might be already known by a significant percentage of the target groups: the data base module of Moodle. To use Moodle even provides additional advantages because of its widespread LMS functionalities. Figure 1. Global structure of learning-outcomes.net As figure 1 shows learning-outcomes.net is not only a web-based data base for collecting, sharing, and improving learning-outcomes but provides space for additional purposes like the common development of a European classification of education or tutorials and even online courses for the competence of writing learning outcomes. Repository of Intended Learning Outcomes – the course The core of learning-outcomes.net is indeed the ILO repository containing the data base, instructional material, templates and other auxiliary information (see figure 2). This course contains everything you need to be able to contribute to the universe of academic learning outcomes. At the moment we can assume that the offered information and tool are neither complete nor perfect. But till the end of 2010 everything shall be completed and of sufficient quality to allow users easy and productive handling of the repository. Upload section – the data base The data base serves as repository on the one hand and as work place on the other hand. With the exception of the PDF file “Module description” (see figure 3 and 4) all content can be modified. Thus particularly the text of the individual learning outcomes (see figure 3: ILO 1 to 9, and figure 5) can be both, translated to all provided languages (at the moment official 23 EU languages and Turkish) and improved in terms of didactic and linguistic quality by different authors. This feature arouses some problems which will have to be solved in the future, at least if the ILO repository will be actively used by as many people as hoped for: quality assurance, accountability, and rights management. How can we make sure that only such modifications of the status quo (of texts and metadata) will be made that improve the quality? Who will moderate or supervise those processes? Does every user get the right to change everything (like in Wikipedia) or have there to be introduced different roles with rather variable rights? These will be the important questions when traffic on learning-outcomes.net begins to expand. Important to mention are the fields going beyond the description of ILOs, namely the question of “e-Learning / e-Assessment” (see figure 3 to 5). The information collected here sheds light on the applicability of a specific module to virtual mobility on the one hand and might extend our horizon of the compatibility of particular learning outcomes with e-assessment. Thus the ILO repository will generate material for further research into this field. Figure 2. Structure of the course “ILO Repository” Figure 3. Structure of the data base “ILO Repository” Module description – overview and allocation The PDF files describing a module (see figure 4) give a general overview of the institutional and didactic features of the module. Thus the institutional information leads to the original author only. Identical or nearly identical modules will certainly occur. Furthermore it is not unusual that one specific module is integrated in different study programs. To meet this ambiguity in both directions the field “mapping to study program(s)” was added. Beyond that this ambiguity is the reason why the study program had to take a back seat in allocation and classification of modules and learning outcomes. The classification criterion according to ISCED (or later hopefully to “EUSCED”) has to be the subject matter in the focus of the module and not of the program; e.g. 342 (Marketing and Sales Management, which is part of social sciences, business and law) is the classification criterion for “Strategic Food Marketing” and not 6201 (Agricultural Economics, which is sub field of Agriculture, forestry and fishery). Figure 4. Structure and example of the file “Module description” ILO files – the atoms of educational design In contrast to the module descriptions the ILO files (see figure 5) are formatted as RTF files which can be processed with most of the existing word processors under nearly all operating systems. Their main purpose is to be changed in terms of quality improvement and translation. You can see the large empty space in the middle of the document which is foreseen for all those expected translations. The table format on the other hand allows for easy integration of modifications respectively corresponding proposals (see figure 6). Figure 5. Structure and example of ILO files One of the most important features of the ILO files is the file name. It carries the bigger part of the classification information necessary for retrieving single ILOs with particular attributes in a Moodle data base avoiding additional fields for metadata. This solution (see tables 1 and 2) is certainly a compromise between quality, usability and amount of work for the authors. Learning Outcome English / modification proposal 1 At the end of the module students are able to apply new research techniques to new situations English At the end of the module students should be able to apply new research techniques to new situations Deutsch / Änderungsvorschlag 1 Am Ende des Moduls sind die TeilnehmerInnen in der Lage, neue Forschungstechniken auf neue Situationen anzuwenden Deutsch / German Am Ende des Moduls sollten TeilnehmerInnen in der Lage sein, neue Forschungstechniken auf neue Situationen anzuwenden Figure 6. Modification of an ILO file Using the file names you can search for single ILOs fitting to each element of the file name, particularly for the subject matter of the module (represented by the module code according to ISCED), the specific quality of a single ILO (represented by the ILO code; for details see table 2), the allocation to the criterion discipline related vs. generic, and the (short) title respectively individual words of it. Table 1: File name convention and example for ILO files. Elements Type Module code ISCED ILO number ILO code Discipline / Generic Short title Example ILO_ M342_ #1_ Lmc32_ DR_ Apply new research techniques Result ILO_ M342_#1_ Lmc32_ DR_ Apply new research techniques Table 2: Classification of intended learning outcomes. Classification of intended learning outcomes Criteria values 1 2 3 4 Competence Domain d,m,p,s d iscipline m ethodical p ersonal s ocial EQF Ability k,s,a,c k nowledge s kills a ttitudes c ompetences EQF Cycle 5 8 5: short cycle 6: bachelor 7: master 8: doctor, phd Competence Stage 1 3 1: novice 2: advanced 3: competent The first draft of the classification scheme shown in table 2 was presented in the GMW paper already mentioned (Csanyi, 2010). Due to its language (German) I will not just refer to this preparatory work, but translate some extracts and illustrate the development since then. “The purpose of (this) classification is a practical one and not an academic one. The applied models of classification have to be subordinated to this purpose and selected accordingly. In addition they have to fit to the concept of EQF which is an already politically established framework for classification on learning outcomes – even if it is not invulnerable from the view of learning psychology respectively competence theory” (translated from Csanyi, 2010, p.7). The vulnerability of the classification scheme mentioned above refers to the criterion “ability”. The draft presented in Csanyi (2010) – at that time naming this criterion “area of EQF” (EQR-Bereich) – strictly sticked to the model of EQF which uses only three options for the classification of abilities: knowledge, skills, and competences. But to completely disregard the emotional / attitudinal dimension of competences would destroy the whole construct of competence from a theoretical point of view and neglect a rather important learning outcome in a number of empiric cases. Thus “attitudes” were re-integrated in the classification model for single learning outcomes. “Classes of competence (Kompetenz-Klasse oder Gruppe): competence is not yet a well-defined construct respectively a homogenously used concept. Again for pragmatic reasons I tend to a further distinction of learning outcomes which is according to my observation the most common one and above that easily comprehensible: personal, social / communicative, methodical and domain related (fachliche bzw. fachspezifische) abilities” (translated from Csanyi, 2010, p.8). In the context of EQF, competence is described in terms of responsibility and autonomy (European Commission, 2008, p. 13). In any social contexts – which represent the overwhelming majority of practical situations – the performance of competence depends on all four classes of competence (which are now named domains for better phonetic discrimination). “Stages of competence (Kompetenz-Stufen): For further distinction of competence levels (Niveaus) according to EQF we propose to apply the first three stages (Stufen) of competence development according to Dreyfus (1981) ranging from novice via advanced beginner to competent problem solver” (translated from Csanyi, 2010, p.8). The reason for introducing this criterion which allows a much more detailed classification than the EQF with its usually 2-years cycles is the need to be able to identify the quality of learning outcomes in the beginning of a study programme when the finally aspired competences will not be fully obtained. The problem with this distinction might be that ILO authors could allocate different stages to identical learning outcomes in the context of different EQF cycles. All in all, the presented classification scheme is an attempt to combine easy operability with the quality of retrieval of individual learning outcomes. Practical experiences of future users will show if the chosen compromise is well balanced or will have to be changed in one or another direction. Summary and outlook The success of the Bologna process is – in several dimensions of its goals – depending on well defined learning outcomes: competence orientation, learner centering, (educational) quality development, (virtual) mobility, and (international) transparency of educational offers. The problem with writing learning outcomes is, that it is a (rather new and) challenging art and that there is no standardized language as central tool for practicing it. On the other hand, each ILO once professionally written and translated can be used in the context of thousands of modules and hundreds of programs in identical form. The web-based ILO repository www.learning-outcomes.net wants to contribute to facilitate shared ILO writing and improving and above that stimulate a collection of e-learning and e-assessment methods. The ILO repository – version 1 is online since mid of October 2010. Its logical and practical structure ispresented in this paper und should be challenged by as many users as possible until the end of 2010. Theincoming feedback will be utilized to design a clearly better version 2. ReferencesAustralian Bureau of Statistics (2001), Australian Standard Classification of Education (ASCED) 2001.Statistics Canada (2001), Classification of Instructional Programmes (CIP), http://stds.statcan.gc.ca/mfs-pde/cipmfs-cpepde-concstat-eng.asp (last visited: 17 October 2010).Dreyfus, Stuart E. & Dreyfus, Hubert L. (1980), A Five-Stage Model of the Mental Activities Involved inDirected Skill AcquisitionEuropean Commission: The European Qualifications Framework for Lifelong Learning (EQF), Luxembourg:Office for Official Publications of the European Communities, 2008,, ISBN 978-92-79-08474-4.Tippelt, R./Mandl, H./Straka, G. (2003): Entwicklung und Erfassung von Kompetenz in der Wissensgesellschaft– Bildungsund wissenstheoretische Perspektiven. In: Gogolin, I./Tippelt, R. (Hrsg.): Innovation durchBildung. Beiträge zum 18. Kongress der Deutschen Gesellschaft für Erziehungswissenschaft. Opladen,S. 349-369.United Nations Educational, Scientific And Cultural Organization (2006), International Standard Classificationof Education / ISCED 1997, May 2006, Re-edition, ISBN 92-9189-035-9, UIS/TD/06-01, © UNESCO-UIS 2006, www.uis.unesco.org.